Skip to content

feat: track api-diff baseline via Renovate and store diffs in docs/apidiffs#2174

Merged
zeitlinger merged 3 commits into
mainfrom
api-diff-baseline
Jun 2, 2026
Merged

feat: track api-diff baseline via Renovate and store diffs in docs/apidiffs#2174
zeitlinger merged 3 commits into
mainfrom
api-diff-baseline

Conversation

@zeitlinger
Copy link
Copy Markdown
Member

@zeitlinger zeitlinger commented Jun 2, 2026

Summary

Fixes the manual-bump drift in the api-diff baseline (#2170) and makes API changes visible in review.

  • Single source of truth. The baseline lives only in the <api.diff.baseline.version> pom property (bumped from the already-drifted 1.5.1 to the actual latest release 1.6.1). mise.toml and api-diff.yml no longer hardcode it.
  • Renovate owns the bump. A custom regex manager tracks the latest published io.prometheus:prometheus-metrics-core on Maven Central and bumps the property on the renovate/api-diff-baseline branch. Because Renovate only proposes published versions, there is no Maven Central propagation race and no post-release workflow or app token needed.
  • Diffs committed to docs/apidiffs/. mise run api-diff syncs the japicmp per-module reports into docs/apidiffs/<module>.diff via .github/scripts/sync-api-diffs.sh, stripping the volatile preamble so files only churn on real API changes. The api-diff workflow fails if they are stale, so every API change shows up in the PR diff.
  • Regeneration on bump. generate-api-diff-baseline.yml regenerates docs/apidiffs on the Renovate branch and pushes it back, mirroring generate-protobuf.yml.
  • docs/apidiffs/** is marked linguist-generated so flint skips it.

Notes

  • The seed diffs are large because 1.6.1 predates @StableApi (the bootstrap noise documented in feat: Add StableApi marker and API diff check #2168). They shrink to near-empty once a release contains the annotations.
  • Like the protobuf flow, a GITHUB_TOKEN push doesn't re-trigger CI — close/reopen the Renovate bump PR to run the api-diff check after regeneration (noted in the workflow).

Validation

  • mise run api-diff (generates the 25 seed diffs; verified idempotent)
  • mise run lint
  • renovate-config-validator --strict, actionlint, zizmor

Closes #2170

Also fixes a pre-existing compat-test break on main

micrometer-compatibility and jmx-exporter-compatibility fail on main (e.g. #2173), unrelated to this change. The compat harness installs local artifacts with -Dmaven.test.skip=true, which skips building the *:test-jar artifacts that the activeByDefault default profiles declare as test dependencies, breaking resolution (e.g. prometheus-metrics-exposition-textformats:jar:tests).

Fixed by deactivating those profiles in the compat install (-P !default), matching what the release task already does (-P release,!default). Verified locally: full mvnw install -Dmaven.test.skip=true -P !default → BUILD SUCCESS.

Reviewer note: the seed diffs show the full stable surface (expected)

The committed docs/apidiffs/current_vs_latest/*.txt list the entire @StableApi surface as additions, not a small delta. This is the documented bootstrap state from #2168, not a bug:

  • The japicmp include filter is @io.prometheus.metrics.annotations.StableApi. The baseline 1.6.1 jar predates @StableApi, so its filtered surface is empty → every annotated class in current code registers as "NEW". The header genuinely compares against the 1.6.1 jar; the annotation filter is what makes everything look added.
  • Verified the filter is correct: only @StableApi-annotated types appear (non-stable classes like CKMSQuantiles/Buffer are excluded; CallbackMetric shows only as an inherited NEW SUPERCLASS reference). It is the same root cause as the breaking-api-change-accepted label on this PR.

It self-corrects after the next release ships @StableApi: Renovate bumps the baseline to that release, the bump workflow regenerates current_vs_latest/ as an annotated-vs-annotated (near-empty) diff, and the archived <new>_vs_<old>/ becomes the first real release diff. So the current files are best read as a one-time record of the initial declared stable API surface.

…idiffs

Derive the api-diff baseline from the latest published release instead of a
hardcoded version, and commit per-module diffs so API changes are visible in
review.

- Move the baseline to the <api.diff.baseline.version> pom property (the single
  source of truth) and let Renovate bump it: a custom regex manager tracks the
  latest published io.prometheus:prometheus-metrics-core on Maven Central, on
  the renovate/api-diff-baseline branch.
- generate-api-diff-baseline.yml regenerates docs/apidiffs on that branch and
  pushes the result back (mirrors generate-protobuf.yml).
- mise run api-diff now syncs japicmp reports into docs/apidiffs via
  .github/scripts/sync-api-diffs.sh (volatile preamble stripped so files only
  churn on real API changes); the api-diff workflow fails if they are stale.
- Mark docs/apidiffs as generated so flint skips it.

Closes #2170
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
The micrometer and jmx-exporter compatibility tests install local
artifacts with -Dmaven.test.skip=true, which skips building the
*:test-jar artifacts that the activeByDefault default profiles declare
as test dependencies. Leaving the profile active broke dependency
resolution (e.g. prometheus-metrics-exposition-textformats:jar:tests),
failing on main and on every PR (e.g. #2173).

Deactivate those profiles in the compat install (-P '!default'), the
same approach the release task already uses (-P 'release,!default').
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger force-pushed the api-diff-baseline branch from b399f53 to 282dddb Compare June 2, 2026 15:34
@zeitlinger zeitlinger added the breaking-api-change-accepted Accept breaking API changes label Jun 2, 2026
Match opentelemetry-java's docs/apidiffs layout and format:
- store per-module diffs under docs/apidiffs/current_vs_latest/<module>.txt
  instead of flat docs/apidiffs/<module>.diff
- keep the 'Comparing ... .jar against ... .jar' header; drop only the
  ignore-missing-classes warning and the semantic-versioning suggestion
- on a baseline bump, archive the previous current_vs_latest as
  docs/apidiffs/<new>_vs_<old>/ for per-release history
Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
@zeitlinger zeitlinger marked this pull request as ready for review June 2, 2026 16:03
@zeitlinger zeitlinger merged commit 3adb890 into main Jun 2, 2026
15 checks passed
@zeitlinger zeitlinger deleted the api-diff-baseline branch June 2, 2026 17:00
zeitlinger added a commit that referenced this pull request Jun 3, 2026
🤖 I have created a release *beep* *boop*
---


##
[1.7.0](v1.6.1...v1.7.0)
(2026-06-03)


### Features

* Add StableApi marker and API diff check
([#2168](#2168))
([768fd3a](768fd3a))
* add typed metric family descriptors
([#2114](#2114))
([9c3b097](9c3b097))
* track api-diff baseline via Renovate and store diffs in docs/apidiffs
([#2174](#2174))
([3adb890](3adb890))


### Bug Fixes

* **deps:** update dependency com.github.ben-manes.caffeine:caffeine to
v3.2.4 ([#2088](#2088))
([144eb61](144eb61))
* **deps:** update dependency io.dropwizard.metrics:metrics-core to
v4.2.39 ([#2139](#2139))
([5817d13](5817d13))
* **deps:** update dependency io.dropwizard.metrics5:metrics-core to
v5.0.7 ([#2140](#2140))
([261c451](261c451))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.0-alpha
([#2126](#2126))
([b62b5d0](b62b5d0))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.0-alpha
([#2127](#2127))
([e11ce3d](e11ce3d))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.1-alpha
([#2132](#2132))
([b09be38](b09be38))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.1-alpha
([#2133](#2133))
([a241c16](a241c16))
* **deps:** update dependency org.apache.tomcat.embed:tomcat-embed-core
to v11.0.22
([#2099](#2099))
([22125c5](22125c5))
* **deps:** update jetty monorepo to v12.1.10
([#2169](#2169))
([ddd3991](ddd3991))
* **deps:** update jetty monorepo to v12.1.9
([#2102](#2102))
([04bee70](04bee70))
* **deps:** update protobuf
([#2129](#2129))
([320538a](320538a))
* Reduce allocations for classic histogram buckets
([#2081](#2081))
([edd160a](edd160a))
* restore legacy suffix compatibility
([#2100](#2100))
([b2ae70f](b2ae70f))
* restore reserved suffix stripping in
`PrometheusNaming.sanitizeMetricName()`
([#2124](#2124))
([2d0f508](2d0f508))


### Performance Improvements

* Refactored sorting to use optimized sort algorithms
([#2161](#2161))
([25b94fc](25b94fc))


### Documentation

* clarify downstream adapter validation requirements
([#2101](#2101))
([ef8c75c](ef8c75c))
* Document OM2
([#2059](#2059))
([45d753c](45d753c))
* document PushGateway shading workaround
([#2106](#2106))
([8ca0eb8](8ca0eb8))

---
> [!IMPORTANT]
> Close and reopen this PR to trigger CI checks.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
jaydeluca pushed a commit that referenced this pull request Jun 4, 2026
…idiffs (#2174)

## Summary

Fixes the manual-bump drift in the api-diff baseline (#2170) and makes
API changes visible in review.

- **Single source of truth.** The baseline lives only in the
`<api.diff.baseline.version>` pom property (bumped from the
already-drifted `1.5.1` to the actual latest release `1.6.1`).
`mise.toml` and `api-diff.yml` no longer hardcode it.
- **Renovate owns the bump.** A custom regex manager tracks the latest
*published* `io.prometheus:prometheus-metrics-core` on Maven Central and
bumps the property on the `renovate/api-diff-baseline` branch. Because
Renovate only proposes published versions, there is no Maven Central
propagation race and no post-release workflow or app token needed.
- **Diffs committed to `docs/apidiffs/`.** `mise run api-diff` syncs the
japicmp per-module reports into `docs/apidiffs/<module>.diff` via
`.github/scripts/sync-api-diffs.sh`, stripping the volatile preamble so
files only churn on real API changes. The api-diff workflow fails if
they are stale, so every API change shows up in the PR diff.
- **Regeneration on bump.** `generate-api-diff-baseline.yml` regenerates
`docs/apidiffs` on the Renovate branch and pushes it back, mirroring
`generate-protobuf.yml`.
- `docs/apidiffs/**` is marked `linguist-generated` so flint skips it.

## Notes

- The seed diffs are large because `1.6.1` predates `@StableApi` (the
bootstrap noise documented in #2168). They shrink to near-empty once a
release contains the annotations.
- Like the protobuf flow, a `GITHUB_TOKEN` push doesn't re-trigger CI —
close/reopen the Renovate bump PR to run the api-diff check after
regeneration (noted in the workflow).

## Validation

- `mise run api-diff` (generates the 25 seed diffs; verified idempotent)
- `mise run lint`
- `renovate-config-validator --strict`, actionlint, zizmor

Closes #2170

## Also fixes a pre-existing compat-test break on `main`

`micrometer-compatibility` and `jmx-exporter-compatibility` fail on
`main` (e.g.
[#2173](#2173)), unrelated
to this change. The compat harness installs local artifacts with
`-Dmaven.test.skip=true`, which skips building the `*:test-jar`
artifacts that the `activeByDefault` `default` profiles declare as test
dependencies, breaking resolution (e.g.
`prometheus-metrics-exposition-textformats:jar:tests`).

Fixed by deactivating those profiles in the compat install (`-P
!default`), matching what the release task already does (`-P
release,!default`). Verified locally: full `mvnw install
-Dmaven.test.skip=true -P !default` → BUILD SUCCESS.

## Reviewer note: the seed diffs show the full stable surface (expected)

The committed `docs/apidiffs/current_vs_latest/*.txt` list the
**entire** `@StableApi` surface as additions, not a small delta. This is
the documented bootstrap state from #2168, not a bug:

- The japicmp include filter is
`@io.prometheus.metrics.annotations.StableApi`. The baseline `1.6.1` jar
predates `@StableApi`, so its filtered surface is empty → every
annotated class in current code registers as "NEW". The header genuinely
compares against the `1.6.1` jar; the annotation filter is what makes
everything look added.
- Verified the filter is correct: only `@StableApi`-annotated types
appear (non-stable classes like `CKMSQuantiles`/`Buffer` are excluded;
`CallbackMetric` shows only as an inherited `NEW SUPERCLASS` reference).
It is the same root cause as the `breaking-api-change-accepted` label on
this PR.

It self-corrects after the next release ships `@StableApi`: Renovate
bumps the baseline to that release, the bump workflow regenerates
`current_vs_latest/` as an annotated-vs-annotated (near-empty) diff, and
the archived `<new>_vs_<old>/` becomes the first real release diff. So
the current files are best read as a one-time record of the initial
declared stable API surface.

---------

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Signed-off-by: Jay DeLuca <jaydeluca4@gmail.com>
jaydeluca pushed a commit that referenced this pull request Jun 4, 2026
🤖 I have created a release *beep* *boop*
---

##
[1.7.0](v1.6.1...v1.7.0)
(2026-06-03)

### Features

* Add StableApi marker and API diff check
([#2168](#2168))
([768fd3a](768fd3a))
* add typed metric family descriptors
([#2114](#2114))
([9c3b097](9c3b097))
* track api-diff baseline via Renovate and store diffs in docs/apidiffs
([#2174](#2174))
([3adb890](3adb890))

### Bug Fixes

* **deps:** update dependency com.github.ben-manes.caffeine:caffeine to
v3.2.4 ([#2088](#2088))
([144eb61](144eb61))
* **deps:** update dependency io.dropwizard.metrics:metrics-core to
v4.2.39 ([#2139](#2139))
([5817d13](5817d13))
* **deps:** update dependency io.dropwizard.metrics5:metrics-core to
v5.0.7 ([#2140](#2140))
([261c451](261c451))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.0-alpha
([#2126](#2126))
([b62b5d0](b62b5d0))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.0-alpha
([#2127](#2127))
([e11ce3d](e11ce3d))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.1-alpha
([#2132](#2132))
([b09be38](b09be38))
* **deps:** update dependency
io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha
to v2.28.1-alpha
([#2133](#2133))
([a241c16](a241c16))
* **deps:** update dependency org.apache.tomcat.embed:tomcat-embed-core
to v11.0.22
([#2099](#2099))
([22125c5](22125c5))
* **deps:** update jetty monorepo to v12.1.10
([#2169](#2169))
([ddd3991](ddd3991))
* **deps:** update jetty monorepo to v12.1.9
([#2102](#2102))
([04bee70](04bee70))
* **deps:** update protobuf
([#2129](#2129))
([320538a](320538a))
* Reduce allocations for classic histogram buckets
([#2081](#2081))
([edd160a](edd160a))
* restore legacy suffix compatibility
([#2100](#2100))
([b2ae70f](b2ae70f))
* restore reserved suffix stripping in
`PrometheusNaming.sanitizeMetricName()`
([#2124](#2124))
([2d0f508](2d0f508))

### Performance Improvements

* Refactored sorting to use optimized sort algorithms
([#2161](#2161))
([25b94fc](25b94fc))

### Documentation

* clarify downstream adapter validation requirements
([#2101](#2101))
([ef8c75c](ef8c75c))
* Document OM2
([#2059](#2059))
([45d753c](45d753c))
* document PushGateway shading workaround
([#2106](#2106))
([8ca0eb8](8ca0eb8))

---
> [!IMPORTANT]
> Close and reopen this PR to trigger CI checks.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Signed-off-by: Jay DeLuca <jaydeluca4@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-api-change-accepted Accept breaking API changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Base the API diff baseline version on the latest release

2 participants